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IMPROVING LANGUAGE ARTS ASSESSMENT OF LANGUAGE 
MINORITY STUDENTS IN COOPERATIVE LEARNING SETTINGS 

Study on Cultural and Linguistic Influences 
on Group Interaction During Problem Solvii^ 

Richard P. Duran and Margaret Szymanski 
CRESST/Univerfdty of California, Santa Barbara 

Abstract 

How can we ensure that new forms of assessment are fair and valid for language 
minority students? This key question undergirds our research on assessing the 
performance of Latino language minority elementary school students engaged in 
a cooperative learning language arts curriculum conducted in Spanish and 
English. This paper describes our strategies for developing performance 
assessments and goes on to present some results from a preliminary study 
implementing these strategies as part of the curriculum for bilingual and 
monolingual children in the third and fourth grades. Our research shows how 
design of individualized performance assessments might be devised based on 
ethnographic observation of children's interaction and the goals for cooperative 
learning interaction established by the teacher. We also describe how analysis of 
children's interaction in cooperative learning activity can show in situ classroom 
assessments among children, helping validate interpretation of the performances 
shown by children on individualized assessment: 

Introduction 

Current efforts to reform student testing call attention to the need for 
testing to become more integral with the everyday classroom instruction 
encountered by students and have begun to include elaborated discussion of 
fairness and equity issues (Darling-Hammond, 1994; LaCelle-Peterson & 
Rivera, 1994; Madaus, 1994), While efforts to develop new assessment methods 
have involved classroom teachers and researchers reflecting on everyday 
classroom practice and curriculum frameworks, the use of applied linguistics 
and qualitative research on classroom interaction to design and evaluate 
assessments has been limited.^ The present report discusses the design and 

1 See Wilkinson and Silliman (1990) for a discussion of this topic from a sociolinguistic 
perspective and Hall, Knudsen, and Greeno (in preparation) for a discussion of this topic from 
the vantage of cognitive science analysis of situated action. 
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pilot implementation of a language arts assessment drawing on these methods 
with third- and fourth-grade elementary school students participating in a 
cooperative learning curriculum known as CIRC (Cooperative Integrated 
Reading and Composition; Madden, Slavin, & Stevens, 1986). The curriculum 
has been adapted for use with both English monolingual and Spanish-English 
bilingual elementary school students. We will refer to this modified 
curriculum as BCIRC following the convention established by the developers of 
the curriculum (Calderon, Hertz-Lazarowitz, & Tinajero, 1991). We wish to 
study, in particular, evidence that children become more competent in their 
performances over the school year in a manner that may reflect the work that 
they do together in cooperative learning activity during the year. 

Thus, we are also concerned in our assessment research with analysis of 
cooperative learning interaction itself. From the theoretical perspective of 
cultural historical psychology, we hypothesize that children first develop 
individualized competence in thinking and language use through social 
participation in academic activities supporting the collaborative acquisition of 
thinking and language skills (Tharp & Gallimore, 1988). As children F5:-ogress 
from little or no competence in a skill area, they begin to display competence in 
an area with the support of more capable others and the teacher. Interaction 
among students and the teacher should reveal this collaborative acquisition of 
skills quite directly. It should be evident in how students in cooperative groups 
approach and negotiate academic problem solving and in how they monitor 
and execute actions completing problem-solving tasks. In essence, the study of 
students' cooperative learning interaction becomes another avenue for 
framing the assessment of children's learning. By studying interaction we 
can make visible the moment-to-moment construction of learning 
performance and how this interaction itself reveals students' and a teacher's 
own assessment of learning. 

Assessment Research Strategy 

Our assessment research and development strategy is general, involves 
four steps, and is recursive. In Step 1 we begin by asking: What expectations 
do teachers establish for children's performance in the cooperative learning 
curriculum, and how might this information be used to identify constructs 
that merit attention in performance assessments of children's learning? We 
rely on ethnographic observation of the classroom in order to discern teachers' 
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goals and -children's interpretation and pursuit of these goals within the 
classroom as a cultural community of practice. 

In Step 2 we ask: What is the nature of the performances expected of 
students given the linguistic and cognitive demands of tasks and materials? 
Based on this analysis, we develop pre and post individuaUzed assessment 
items that can capture students' mastery of intended performances in a target 
performance area over the covirse of the year. We then proceed to analyze and 
interpret change in children's performance from pre- to post-assessments and 
derive hjrpotheses about forms of cooperative group problem solving that might 
bear evidence about changes in children's performance. 

In Step 3 of our program, we go on to conduct detailed discourse analytic 
and conversation analysis study of video tapes of children's cooperative 
learning interaction collected during the intervening period between 
individualized pre- and post-assessments. Our detailed analyses of the 
teacher's and children's interaction reveal evidence on how children's 
cooperation might address the performances expected of children by the 
teacher and whether this evidence corroborates the performances required by 
individualized pre- and posttests. The analyses of interaction, further, make 
visible forms of cooperative learning performance, capturing additional forms 
of face-to-face assessments enacted by students (and the teacher) that are 
represented by or may go beyond the individualized pre- and post-assessments 
presented to students. 

In Step 4 of our assessment program, we reflect on new ways to direct the 
design of assessments and ways to make the previous assessment results and 
new assessment design relevant to teachers and children. 

The BCmC Curricultim2 

The BCIRC cooperative learning curriculum is built around "Treasure 
Hunt" story reading units, available in Spanish as well as English. 
Regardless of language, students in mixed-ability cooperative groups of 4 to 5 
pupils participate in the same core set of Treasure Hunt activities. Core 
activities include a discussion of target vocabulary terms helping to preview 
the content of a story, silent and then dyadic oral reading of the first part of a 



2 Implementation of the BCIRC curriculum has been supported by a research grant from the 
Center for Effective Schooling of Disadvantaged Students, Johns Hopkins University. 



4 



CRESST Final Deliverable 



story, and group discussion and individualized written answering of story 
questions for Part 1. The final question for Part 1 of a story is a prediction 
question. The second part of a Treasure Hxint proceeds in the same way as the 
first part, except that there is no further vocabulary introduction and there is 
no built-in requirement of a final prediction question. Additionally, once Part 2 
is completed, students are asked to discuss in groups and individually 
compose story-related essays. Other activities distributed across days include 
student coordinated testing of vocabulary spelling and meaning, and story 
recall. A complete Treasure Hunt cycle takes a minimum of four days, but 
can be extended to up to two weeks if teachers decide to build in supplemental 
instructional activities. 

Step 1: Ethnographic Observation and Selection of Assessment Constructs 

The BCIRC curriculum is complex. Our identification of constructs for a 
performance assessment has been deliberately selective. While we could have 
easily opted for a portfolio-level collection and scoring of products generated 
over the course of an entire Treasure Hunt or series of Treasure Hunts, we 
have instead, focused thus far on specific linguistic skills that teachers expect 
of students in their performance of Treasure Hunt question discussion and 
answering activities. We have chosen this strategy for the present because we 
wish to focus on particular kinds of performance that might be reflected in 
teachers' curriculum goals, and that might be studied in detail, to reflect 
bilingual students' development of critical academic language skills in 
English. We elected question discussion and answering activities and how 
students generate written answers to questions because all our teachers placed 
emphasis on this issue in guiding students' cooperative group and individual 
performance. In addition, the issue is well-recognized in the sociolinguistic 
and literacy research as significant to children's development of an academic 
written discoiirse style that requires transforming children's existing 
knowledge of how to interact in oral conversation (Collins & Michaels, 1986; 
Gumperz, 1986). 

The issue of answering questions in complete sentences in academic 
assignments is also important because it applies regardless of bilingual 
children's language of instruction. The skill in question is part of the common 
underlying academic language competencies expected of students regardless 
of language (Cummins, 1984). As such, it also raises the possibility that once 
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' stiidents attain competence -in the skill in the first language, attaining 
competence in the skill in the second language can build on the same 
competency in the first language. 

We have broken down the design of an individualized performance 
assessment of complete sentence answering into three subareas: (a) ability to 
"echo" a question in the answer reply, regardless of whether a reply is a 
complete sentence; (b) writing the answer as a complete sentence regardless of 
echoing; and (c) expressing acceptable and anticipated meaning in the content 
of the answer. In this paper we report our progress on only the first of these 
three subareas. Our selection of question echoing with complete or incomplete 
question replies was sensitive to the performance criteria established by 
teachers. Our ethnographic observations ft-om the previous year revealed that 
teachers expended considerable attention in providing students with direct 
instruction and modeling in this skill area. Teachers explicitly told students 
that they were to start out group discussion of how to answer a Treasure Himt 
question by deciding what portion of a question had to be included in the initial 
firaming of a question answer. Students were then instructed to fill in the 
remaining part of the answer based on information requested by the echoed 
portion of the reply. In addition to modeling this process for students, teachers 
also introduced underlining of the portion of the question to be included in the 
reply as an aid for students to keep track of what was expected of them. Some 
teachers had students use a yellow marking pen for this purpose. 

We have distinguished assessment of students' simple ability to produce 
complete sentence answers (subarea b) fi:om their ability to produce complete 
and incomplete sentences with differing degrees of echoing a question in a 
reply (subarea a). Even though teachers emphasized that a good question reply 
would begin with an echoing of the question, students independently might 
first attempt to master writing an answer that was a complete sentence 
without worrying about echoing. In our assessment work we wanted to 
handle this matter by developing a separate scoring system for their answer 
replies. Also, as work proceeds, we will investigate alternative syntactic 
models for coding students' replies that will allow us to characterize specific 
ways in which competence in echoing combines with competence in writing a 
complete sentence answer and with the semantic acceptability of an answer. 
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Step 2: Design and Administration of Pre- and Post-Assessment Item and 
biterpretations of Performance 

Our design of individualized pre- and post-assessments of echoing drew 
on a linguistic study of the structure of questions encountered by students in 
the BCIRC curriculum. We took into consideration major grammatical 
question types encountered by students on Treasure Hunts. Those identified 
included "Wh" ("who," "when," "why," "what") and "how" questions in English 
and their corresponding analogs in Spanish. We also examined questions 
according to their clausal and phrasal complexity (e.g., occurrence of 
subordinate and compourd clauses, and occurrence of adverbial and adjectival 
phrases). 

Based on these analyses, we constructed pre- and post-assessment items 
in Spanish and English drawn from our project's pool of Treasure Hunt story 
passages and questions not previously used by children. Each test consisted of 
a several-paragraph story segment divided in two parts, each accompanied by 
four questions. 

The pre-assessment was administered in late October and the post- 
assessment in early May. The pre-assessment was not a "pure" pre- 
assessment in that children had begun to be exposed to BCIRC at the start of 
October. Administration of pre-assessments after introduction would be 
clearly undesirable if we had intended a formal evaluation of BCIRC effects. 
We were not overly concerned with start-up of BCIRC just prior to our pre- 
assessment because of the pilot character of our work and because we were not 
formally evaluating BCIRC. We beUeve that prior exposure to BCIRC before 
the pre-assessment had negligible effects on assessment performance because 
the BCIRC curriculum is so complex that teachers had not had an opportunity 
to expose students to specific question answering strategies so early on in 
introducing the curriculum. 

Thfi story segment and question items on the pre- and post-assessments 
were identical in format to those encountered by students in everyday 
classroom Part 1 and Part 2 Treasure Hunt activities. However, on the pre- 
and post-assessments, students were administered these materials as 
individual assignments to be completed within a one-hour time period and 
without recourse to cooperative group discussion of questions and answers. 
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Pre- and post-assessments were administered to 2 third-grade bilingual 
classrof ms. One classroom was a bilingual Spanish-English (BCD classroom 
(N=16) and the other was a monolingual English classroom with some 
students from Spanish language backgrounds (ECL) (iV=23). The assessments 
were administered to the students in their language of instruction. Our 
analysis focused on the students' answers to four pre-assessment and four 
post-assessment questions that were matched for their grammatical similarity 
(Appendix 1).3 

Students' answers for the four pre-assessment and post-assessment 
question pairs were analyzed for the ways in which students were seen to be 
using the written question as a resource for their written answer. Six 
response categories were formulated: 

1: No Echo 

2: Elliptical Echo 

3: Minimal Echo 

4: Unintegrated Echo 

5: Echo with Syntactic Repetition 

6: Complete Echo 

These categories are not intended to represent an interval scale of 
developmental competence in written discourse. Rather, they represent our 
judgment of ordinal-level differences regarding students' ability to integrate 
the structure of the prompting question into their written answer. Whi?e our 
assessment of echoing is not the same as an analysis of the grammatical 
completeness of question responses, our examination of data suggests that 
students scoring high on the echoing scale must, of necessity, be producing 
grammatically complete sentence answers when they incorporate parts of the 
question's syntax and lexicon into their answer. We now present a discussion 
of the meaning of echoing response categories. 

LlJe stude^ntt T? ^.'"'''^ ^T. '^"^^ ^PP^P^ate to the reading level of the third- 

s', .The pretest story is a bilingual folk tale from Mexico called The Cuckoo's 

Th^Bo^oftZ'rL'ii Z ^""^ ^^^^'^t ^""^ t'-^^l^t^d by Daisy Kouzel. The posttest story, 
reTpfr^h l ff n ^^f "^""^ ^^^^^^^ translated into Spanish by our 

^«m^n.r« ? ^ "^"^ •'"^"^^ 'h«y contained canonical three-part story 

ftl^Zr.f. P"'^^^'"^' ters a problematic situation, this problem culminates as the 
story events unfold, and the problem is resolved. 
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question. Typically "no echo" responses are ^° jr'f rf.tely 
in quality. That is. the question "Where are ^^J^ J^^.,,, 
answered in conversation w.th "to school. A B"-^" 
response (I am going to school) in conversation would a roarked 
Examples of "no echo" responses for the pretest pred.ct.on question 

What do you think the loud cracUing noise that the owl hears is? 
que crees que se debe la faerte crepitacion que oye la lechuza? 

were: 

1014- porquo ya venia el fuego a quemar las semiHas 
(because the fire already came to bum the seeds) 

127: because the fire 

The part of the question that would count as --i^^^^^^^^^^^ 

question, is "The loud crackling noise is . . . - By '^^^^ \ „„t 

Ldent enables the answer to stand alone ^^^^'^'^^ .^elf,. The 

depend upon reference to a prior prompt (in this case tn q 

adUal noun phrase "that the owl hears" is an -^"J^Ze.t 101 

is the phrase "I think." As the data show, neither ^^ ^^^ „ 

refer in any way to how "the fire" or J^^^^e reader of 

related to the crackling noise elicited in the question. As a result, t 

the answer does not have any way of knowing the significance of the answer 

"fire-" , . t„ 

"No echo" responses were relatively infrequent as — ^^^j^ 
overwhelmingly rely upon their conversational skills ^--^^^g^^ 
the prompting question in their written answers. The seco^^ S 
•elliptical echo," contained answers with a ^^^^^^^^^'^J^^^ ^^^.^ 
.yutax of the base question for its comp^eten. . ^ n the w , 
answer elliptically refers to the question. Some 
pretest question 2: 
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What did Chac ask the birds to help him do? 

^Que les pidio Chac a los parjaros que le ayudara a hacer? 

were: 

108: que al amanecer que fueran adonde estaban las floras y 

agarraran todas las semillas y las pusieran cerca del bosque 
(to at dawn to go to where the flowers were and gather all the 
seeds and put them near the forest) 

110: recojan las semillas porque si no ya no vamos a tener comida. 

(gather the seeds because if not then we are not going to have food) 

141: plant seeds for a new crop so the birds don't go hungry. 

147: to help him plant seeds before the fire god comes. 

Each of these responses shows an orientation to the question's syntactic 
structure. The information solicited by the question is the action that the Chac 
asked the birds to do. This corresponds to the following echo answer frame 
(optional phrases are in parenthesis): 

a. Chac asked the birds to (help him) + infiniti ve 

b. Chac les pidio a los pajaros que le a5aidara a + infipitive 

or 

c. Chac leb pidio a los pajaros que + suh junctiv fi verb phrase 

Students 141 and 110 begin their written answer with the complementary 
infinitive verb form that visibly orients the answer as a continuation of the 
question frames shown above in (a) and (c). Student 141's response is a 
continuation of the infinitival verb phrase prompted by the question (to 
infinitive), whereas student llO's response shows a dependency on a prior 
clause in its subjunctive conjugation. Further evidence of the attached-to-the- 
question nature of elliptical responses is found in responses 108 and 147. Here, 
the students have incorporated the initial phrase marker of the question's 
s;;Titax. Response 108 incorporates the "to" that heads an infinitival verb phrase 
and 147 repeats the "que" that characterizes it as a subordinate verbal phrase. 
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' '' When sttidents refer to the question's syntax and repeat its verbal i 
lexical form^, it became more clear that their writing process included 
explicit reference to the prompting question. When one or more phrases fi 
the question are repeated in the answer, a "minimal echo" response y 
coded. Some "minimal echo" responses for pretest question 5: 

What did the owl see when he flew above the trees? 
^Que vio la lechuza cuando volo sobre los arboles? 



were: 

106: Vio el dios de fuego corriendo todos los lados. 

(He saw the Fire God running ail over the place.) 

115: La lechuia miro mucho humo. 

(The owl looked at a lot of smoke.) 

141: The fire god burning the trees. 

148: The fire god running through the trees with the torch. 



These responses show a minimal echo of the question in their repetition of 
either the subject (the owl/la lechuza) as in 115, the verb phrase (saw/vio) on a 
lexical and/or temporal level as in 106, or a direct object (the trees) as in 141 
and 148. 

It should be emphasized that the coding scale is descriptive of categories 
that arose upon inspection of a particular aspect of the students' written 
responses, that of the repetition or incorporation of the question's syntax and 
lexicon in the answer's syntax. Therefore, a minimal echo response is not 
intended to necessarily allude to the students' abiHty to v/rite a complete 
grammatical sentence. Nonetheless, as mentioned earher, we did find a 
strong correlation to exist between the grammatical completeness of the 
answer and the echo of the question. 

Some students incorporated selected parts of the question's syntax in their 
answer; however, the syntax was not fluid or smooth. These echo answers 
were described as "unintegrated echoes." Consider posttest question 5 and the 
targeted part of the question to be included in the question shown below: 
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What does the ujigami tell the merchant to do? 

The ujigami tells the merchant + infinitival verb phrase 
^Que le dice el ujigami al comerciante que haga? 

El ujigami le dice al comerciante que + subjunctive verb phrase 

The unintegrated echo responses to this question showed the students to be 
relying upon two strategies to write the answer. First, they are selecting parts 
of the question with which to begin and frame the answer. Second, in a task 
separate from writing the answer frame, the students "add on" the answer to 
the answer frame. The unintegrated characteristic of the completed written 
answer lies in the students' use of connectors to achieve the link between the 
separately formulated answer frame and answer. Some unintegrated echo 
responses to this question are: 

106: El ujigami le dice al comerciante que haga es que su hija case con 
Taro. 

(The ujigami tells the merchant to do is that his daughter marries 
Taro.) 

152: I think that the ujigami told the merchant to do was that he must 
make his daughter marry the boy of the Three-Year Nap or he 
will turn her into a pot. 

Clearly, the students are repeating the question in their answer, but their 
strategy to do this involves the use of the connecting verb phrase "is that/es 
que" which transforms the answer frame into the subject of the sentence. That 
is, from a complex (2-clause) question, the student simplifies the syntax of the 
answer through the use of this connector that converts the answer into a 
simple (1-clause) Subject-Verb-Object format corresponding to Answer Frame- 
Connector-Answer. 

Another strategy the students were seen to rely upon in using the question 
to formulate their answer was the contiguous placement of a would-be 
elliptical response with the answer frame. This results in an "echo with 
S3nitactic repetition." Some echo responses with S3mtactic repetition for posttest 
question 5: 
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What does the ujigami tell the merchant to do? 
iQue le dice el ujigami al comerciante que haga? 

are: 

103: Le dice el ujigami al comerciante que haga que case a su hija con 
Taro. 

(The ujigami tells the merchant to do to marry his daughter with 
Taro.) 

107: Le dice el ujigami al comerciante que haga case a su niiia con 
Taro. 

(llie ujigami tells the merchant to do marry his daughter with 
Taro.) 

121: The ujigami tells the merchant to do by saying to let Taro get 
married with his daughter. 

A complete echo response was one in which the base question's syntax 
and lexicon are used to appropriately frame the answer. The responses below 
constitute complete echo answers to posttest question 5 listed above. 

101: El ujigami le dijo que se casara con su hija y Taro. 

(The ujigami told him to marry his daughter and Taro.) 

132: The ujigami told the merchant to let Taro and his daughter get 
married. 

155: The ujigami told the merchant to make his daughter marry a fine 
lad named Taro. 

Also within the category of complete echo, the students may have incorporated 
an adequate echo of the base question with a manipulation of the question's 
syntax. Most commonly, this manipulation involves the use of pronouns for 
full noun phrases or synonymical lexicon. Some manipulated complete echo 
responses for the posttest question: 

Why do you think no one has called Taro "The Boy of the Three-year Nap" 
in a long time? 

^Por que crees que por mucho tiempo nadie ha llamado a Taro "El 
muchacho de la siesta de tres anos"? 
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are: 

118: Taro is not called that anymore because he does not take naps. 

132: I don't think anyone has called him that because of the work he's 
been doing. 

In responses 118 and 132, the name Taro was called, "The Boy of the Three- 
year Nap," is replaced by the referent "that'' in a manipulation that transforms 
the question's syntax into an answer. 

Contingency Table Results 

Appendix 2 displays two-way contingency tables summarizing the pre- 
and post-assessment echoing performance of students from the two target 
classrooms on each of the four paired question sets. These tables graphically 
summarize growth in students' ability to echo questions in written answers to 
questions over the school year. The levels of the columns represent the echoing 
score (1 to 6) earned by students on their written response to a pre-assessment 
question. The row levels represent the echoing score earned by students on 
their written response to a paired post-assessment question. The first set of 
tables, labeled "Simon," corresponds to students assessed in Spanish on both 
the pre-assessment and post-assessment. ("Simon" is the surname of the 
teacher of this bilingual classroom where instruction was in Spanish at the 
start of the school year.) The second set of tables, labeled "Hug," corresponds to 
students assessed in English on both pre-assessment and post-assessment 
("Hug" is the surname of the teacher of this monolingual Enghsh classroom). 

The bivariate distribution of pre- and post-assessment echoing scores for 
students assessed in Spanish (Simon's classroom data) shows evidence of 
development in students' echoing across the school year for all four question 
pairs. Almost all of the table entries are at or below the diagonal, indicating 
that students maintained or exceeded their pre-assessment echoing scores on 
the post-assessment. The bivariate distribution of pre- and post-assessment 
echoing scores for students assessed in English (Hug's classroom data) also 
shows development in students' ability to echo questions in their written 
responses to questions across the school year. The results for students in the 
monolingual English classroom do not show as much dramatic increase as 
those for students assessed in Spanish and attending a bilingual classroom. 
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More students- in the monolingual English classroom started out the year with 
strong ability to echo questions in written answers to questions. 
Step 3: Interaction Analysis of Cooperative Learning Showing Discussion of 
Echoing by Teacher and Students 

The results of the pre- and post-assessment show clear evidence of 
changes in children's capacity to echo questions in written answers on 
individualized assessments. In addition, and independently, BCIRC students 
under investigation can be seen to interactionally achieve the formulation of 
the written answer to a question within their peer group cooperative learning 
activities. Through an interaction analysis of their peer group activity, we 
have isolated several episodes that illustrate the teacher's instruction in the 
peer group to incorporate the question into the answer and the students' 
application of this practice. Example 1 shows the teacher coming into a peer 
group and explicitly outlining the steps in the question-answering process (see 
Appendix 3 for transcription conventions). 

(1) [1/26/93 f- 321 

Teacher: okay, let's do the first one, 

{R>} what are some of the new noises 
in Peter's house {<R} 

what is the part that we're gonna to underline, 
Albert: *^the [new noises° 
Alicia: [*^the new noises*^ 

[SOME 

Teacher : [ Enunanue 1 , 

Albert: SOME, some of the new [noises 
Teacher: [GOOD, 

some of the new noises, 

so underline that, 

some of the new noises in Peter's house ARE, 
((points to Albert)) what's your answer. 

Moments later, the teacher guides the students' answering of the next 
question. In Example 2, her strategy is sUghtly altered and the underlined 
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part of the -question is not explicitly elicited or referred to until this frame has 
been conversationally formulated by the "start** of the answer. 

(2) [1/26/93 f. 33] 

Teacher: {R>} why do you think [Peter] is getting angry, {<R) 
Alicia: [Vhy] 
Teacher: how's that answer gon[na start,?] 
Albert: [°do you^] 

Jorge: because 

Albert: do YOU think Peter is angry, [»yes he] 's angry« 
Teacher: [why,] 
Teacher: why do you think- 
Jorge: because the wish [bo-] 

Teacher: [wait] a minute, when I say 

WHY do you think, what is your answer going to be,? 

Albert: why do you think Peter's angry=cuz all the an- animals 
are doing, (like making noise] 

Teacher: [»wait a minute , wait«] a minute, why 

do you think (.) what [is your answer going to be,?] 

Emmanuel: [because we are t(h) inking] 

Teacher: start the answer, just start the answer. 

Teacher: why do you think 

Emmanuel: ((raises his hand)) oh, I know, because i- [be]cause 
Teacher: U^o] 
Jorge : I think 

Teacher: ah, you hear how he started the answer? (0.2) what did he say? 
Albert : ah 
Emmanuel: I think 

Teacher: I think, so that's how you're gonna start, I: think, 
and you're gonna underline, I think Peter is getting 
angry becau:se ((underlines)) 



Example 3 illustrates how Davina relies upon the underlining strategy to 
guide her through the answering of the question "How do we know that 
dinosaurs ever existed?** In Davina's highlighted turn, she has trouble 
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formulating aii' answer to the question. As a result, she resorts to the 
underlining task. as a part of the answering task ("what are we gonna 
underline"). After selecting the answer's frame from the question's syntax, 
Davina orients to the answering task, ("an the answer"). 



(3) [5/19/93 f. 38] 

Davina: kay read it 

.hh how do we kno:w that (.) dinosaur (0.1) ever 
existed? 

(1.2) ((Davina raises hand)) 

uhm th«y ov«r existed (.) no, be;cu:sz (0.1) 
yeah that- who who uhm (.) uhm, what's gonna 
be the thi- (.) what are we gonna underline,? 

(we know that dinosaurs ever existed] 
(we know that dinosaurs ever existed because] 
(we know that dinosaurs ever existed because] 
there was 
(0.2) 

Yolanda: nothing, [du::h.] 

Davina: ((an') the] answer. 



Yolanda : 



Davina: 



George : 
Davina : 
Yolanda : 



Example 4 shows the students relying upon their conversational skills to 
orally answer the question in an elliptical manner. 

(4) [5/19/93 f. 39] 

George: {R>} how big was a bria[tric] ((shakes head 'no')) 
Sonia: [hah?] 
(0.1) 

George: [brontosaurus ] {<R} 

Davina: [brontosaurus] 

George: [seventy five feet tall] 

Davina: [seventy five feet tall] huh,? 

Yolanda: (linto microphone)) seventy fi:ve feet tall 
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Example 5 below shows the interaction as it proceeds. The students are 
seen to reformulate or reframe their answer into a complete sentence syntax 
by using the question in their answer. Davina initiates this task by prompting 
the others to focus on "the thing" or the answer's frame. Both she and George 
alternately propose candidate answer frames until Sonia produces what would 
be a complete manipulated echo on the last line of the transcript. 



(5) [5/19/93 f. 39] 

Davina: and the thing is uhm 
George: ho: :w was a 

Davina: how big wa:s a: brontosaurus 
George : °no 

Davina: so it's gonna be the biggest was a binanasaurous , 
[it was] 

George: [no::], not lilce that, how Wcis a brontosaurus 
(0.6) 

Sonia: how BI:G was it,? 

Davina: niamber three [go] 

Sonia: ((raises hand)) [»I] know I know« 

Yolanda: ((into microphone)) how BIG was it 

Sonia: it was a (.) seventy five foot long. 



Step 4: New Directioiis in the Design of Assessments Relevant to Teachers and 
Students 

Our study has shown that the form of the students' written products is the 
result of acquired courses of action as instructed by the teacher. Namely, the 
students incorporate parts of the question into the answer as a result of the 
teacher's instructions to "underline" the part of the question that frames the 
answer. Skills assessments then should be formulated to reflect the 
interactions that teacher and students have as they construct teaching and 
learning activity. 

One innovation in the assessment of interactions as they reflect and 
produce particular individual students' products is to analyze students' 
interactions in relation to the product that resulted from the interaction. 
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Further, we need to examine more carefully how students' interactions 
represent their own self-assessment. Our assessment design needs to capture 
this facet of interaction. This assessment design, if tied to feedback to the 
student by a teacher shown interaction analyses, would potentially enable 
students to better use their peers as resources in the completion of an 
individual task. What this means for teachers is that they can be trained to 
"see" productive interactions and to evaluate for themselves the dynamic of 
cooperative learning peer groups. 

We envision in the next phase of our research changing our pre- and post- 
assessment procedures so that students are allowed to interact while 
answering assessment questions. By analyzing students' interaction on the 
pre- and post-assessments we can gain further evidence of how their 
conversational and discourse skills might contribute to their written answers 
over the school year. Further, we could also compare the pre- and post- 
assessment performance of these students with that of other students who 
follow the current pre- and post-assessment procedure. 

In addition, we are undertaking steps to develop a more comprehensive 
linguistic model of question answering. In this report we have examined one 
facet of competence in writing answers to questions. We are currently 
developing a more sophisticated linguistic model of question answering that 
will also assess students' competence in integrating echoing with complete 
sentence answers and appropriate semantic responses to questions. 
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Appendix 1: Pre- and Post-Asses*?ment Question Pairs 

!• pretest prediction question: 

What do you think the loud crackling noise that the owl hears is? 
^ que crees que se debe la fuerte crepitacion que oye la lechuza? 
posttest question prediction: 
What do you think Taro's plan is? 
^Cual crees que es el plan de Taro? 

2. pretest question 2: 

What did Chac ask the birds to help him do? 

^Que les pidio Chac a los pajaros que le ayudara a hacer? 

post-test question 5: 

What does the ujigami tell the merchant to do? 
^Que le dice el ujigami al comerciante que haga? 

3. pretest question 6: 

Why do you think the cuckoo worked so hard to save the seeds? 
iPoT que crees que el cuco trabajo tanto para salvar las semillas? 
posttest question 7: 

Why do you th. ak no one has called Taro "The boy of the 
Three-Year Nap" in a long time? 

iPoT que crees que per mucho tiempo nadie ha llamado a Taro 
"El muchacho de la siesta de tres afios"? 

4. pretest q[uestion 5: 

What did the owl see when he flew above the trees? 

^Que vio la lechuza cuando volo sobre los arboles? 

posttest question 1: 

Where do Taro and his mother live? 

^Donde viven Taro y su madre? 
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APPENDIX 2 



Simon Pre-test Q2 to Post-test Q5: What, Compl 
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Post-Test 



Simon Pre-test Q4 to Post-test Q4: What, Compl 
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Post-Test 



Simon Pre-test Q6 to Post-test Q7: Why, Complex 
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Post-Test 



Simon Pre-test Q5 (What) to Post-test Ql (Where): Simple 
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APPENDIX 2 (CONTINUED) 

Hue Pr e^test 02 tn Post^test O ^^ What. Complex 
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Appendix 3 
Transcription Conventions 



[ ] overlapping speech 

( ) unsure hearing 

(( )) transcriber's and analyst's comments 

{R>} {<R} reading aloud 

: lengthened pronunciation 

? final rising intonation 

, listing intonation (e.g. more is expected) 

final falling intonation 

(.) micropause 
(0.2) two tenths of a second pause 

MY stressed pronunciation 

= latching of speaker's utterances 

truncation (e.g. what ti- what time is it?) 
°bye softly spoken 

>goodbye< rapidly spoken in relation to surrounding 
.h in breath 
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